Prevention Science 
https://doi.org/10.1007/s11121-019-01035-z 


Comprehensive Cost Analysis of First Step Next for Preschoolers Cheek fo 
with Disruptive Behavior Disorder: Using Real-World Intervention 
Data to Estimate Costs at Scale 


Andy J. Frey' - Margaret R. Kuklinski? - Kiersten Bills' - Jason W. Small? - Steven R. Forness* + Hill M. Walker*? - 
Edward G. Feil? - John R. Seeley” 


© Society for Prevention Research 2019 


Abstract 

While the long-term societal costs for youth with disruptive behavior disorders are well documented, there is a dearth of 
information about the comprehensive costs of implementing even the most well-regarded early intervention programs, and the 
costs of scaling effective interventions are even less well understood. This study estimated the costs of delivering and dissem- 
inating First Step Next (FSN), an established tier two school-based early intervention, in preschool and kindergarten settings, 
including the training and ongoing technical assistance that support sustained, high-quality implementation. Using the 
Ingredients Method, we estimated (a) the per student costs of implementation, (b) the incremental cost of offering FSN to an 
additional student, and (c) the cost to disseminate FSN to 40 preschool and kindergarten students, including a sensitivity analysis 
to examine potential areas of cost savings. The per child cost to implement the FSN intervention with 29 triads in two cohorts was 
$4330. The incremental cost per additional student was only $2970, highlighting efficiencies gained once intervention infra- 
structure had been established. The cost of disseminating the intervention to a single cohort of 40 students was $170,106, or 
$4253 per student. The range in sensitivity analysis was $3141—$7829 per student, with variability in personnel wages having the 
greatest impact on cost estimates. This research expands on existing literature by providing a more comprehensive understanding 
of the cost of effective disruptive behavior interventions based on real-world implementation data, using these data to estimate 
dissemination costs, and showing how dissemination costs are particularly sensitive to personnel wages. 
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Disruptive behavior disorders involve a constellation ofsymp- —_ persistent impairment and comorbid psychiatric disorders 
toms including oppositional, defiant, or aggressive behaviors, such as attention deficit hyperactivity, depression, and anxiety 
and are one of the most prevalent disorders in young children _ disorders (Burke et al. 2010). Despite the prevalence of these 
(Egger and Angold 2006). Left untreated, they may resultin disorders, there is a dearth of comprehensive economic 
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evaluation of early preventative interventions (Aos et al. 
2004; Lynch 2004), including fundamental cost analyses that 
can help support high-quality implementation by detailing re- 
sources needed to achieve intervention impact and their asso- 
ciated costs. This study adds to existing literature by pro- 
viding a comprehensive cost analysis of First Step Next 
(FSN), an empirically based early intervention offered at 
the beginning of a child’s school career to improve school 
success, based on data from an efficacy trial. In estimating 
total, average, incremental, and dissemination costs, the 
study provides a more extensive set of cost-related infor- 
mation for intervention implementers, developers, and 
program planners than is generally found in prevention 
cost studies. 


First Step Next Intervention 


The First Step early intervention program (Walker et al. 1997), 
along with its recent revision (FSN; Walker et al. 2018), is an 
established school intervention with a parent engagement 
component. The intervention, which is designed to be imple- 
mented with one child per classroom who is screened for 
having the highest risk in each preschool classroom for dis- 
ruptive behavior, consists of three major program tasks (social 
skills instruction, green card game, home-school connection) 
implemented across four implementation phases (preparation, 
coach, teacher, maintenance). For the social skills instruction 
task, the coach delivers a curriculum, which consists of seven 
super student skills. The coach typically meets with the stu- 
dent outside of the classroom once per day for 7-10 consec- 
utive days, for approximately 10 min each session. The green 
card game is played in the classroom with all students. While 
the game is being played, a red-green card is displayed to cue 
the target child that he or she is displaying appropriate behav- 
ior. When the card is on green, he/she earns points toward a 
group contingency reinforcement activity, delivered immedi- 
ately after the game concludes. The game is initially played for 
only 20 min and gradually increases over the course of the 
intervention. The coach runs the game for approximately 
10 days, and then responsibility for running the game is grad- 
ually turned over to the teacher. The teacher phase lasts for 
10 days followed by a maintenance phase (also 10 days) in 
which the coach is involved on an as needed basis. 

The home-school connection task includes an initial meet- 
ing with the parent, teacher, and coach. During the coach 
phase, the coach also meets with child’s parent (or caregiver) 
to explain the parent role in the home-school point system and 
in providing parenting strategies that may help the child mas- 
ter super student skills in the home setting. The parent is also 
asked to sign and return a home-school note daily and engage 
the child in a 5- to 10-min reinforcement activity after school 
on days the child wins the green card game. The preschool 
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intervention takes approximately 2 months to complete. A 
similar but abbreviated “booster” intervention takes place dur- 
ing the initial weeks of kindergarten to help solidify gains 
made in the preschool year. A complete program description 
is provided in Walker et al. (2018). 

Note that two recent large-scale First Step randomized 
controlled trials (Feil et al. 2014; Walker et al. 2009) and 
a large-scale effectiveness study (Sumi et al. 2013) have 
all produced moderate to large effect sizes on a range of 
important school-related outcomes, such as social compe- 
tency, problem behavior, and academic engaged time. The 
recent revision, FSN, keeps First Step core features but 
makes them more user friendly and updates or streamlines 
its components (Walker et al. 2018). 


Problems and Prospects in Cost Analyses 
of Disruptive Behavior Interventions 


Providing agencies with comprehensive cost estimates of 
tested and effective interventions can help ensure imple- 
mentation capable of achieving impact (Proctor et al. 
2011; Raghavan 2012; United States Department of 
Education 2017). Economic evaluation guidelines 
(Crowley et al. 2018; National Academies of Sciences, 
Engineering, and Medicine 2016; Sanders et al. 2017) 
recommend that cost estimates reflect the opportunity cost 
of providing the intervention, valuing all resources, even 
those that are not paid for directly. However, many cost 
studies do not fully capture the training and capacity 
building, ongoing implementation and technical support, 
and overheard that are needed for high-quality implemen- 
tation, limiting their utility for program planners and other 
stakeholders. Examples include cost estimates for two of 
the most effective interventions for disruptive behavior 
problems in young children, the Incredible Years 
(Menting et al. 2013) and Parent Child Interaction 
Therapy (PCIT; Zisser and Eyberg 2017). A cost analysis 
of the Incredible Years with group parent and child train- 
ing components (Washington State Institute for Public 
Policy [WSIPP] 2018; Foster et al. 2007) yielded a cost 
of $3970 per family (2015 dollars) inclusive of training, 
intervention, materials, childcare, and transportation costs, 
but whether ongoing support for interventionists was in- 
cluded is not clear. WSIPP estimated PCIT at an average 
cost of $2993 per target child (2017 dollars), but which 
key intervention activities were included is also not clear. 
Neither study included overhead costs. 

Total and average cost estimates provide important basic 
information about resource requirements and associated costs, 
and they also help delineate how many participants can be 
served within a given budget allocation. As efficacious inter- 
ventions are implemented more broadly, however, additional 
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cost-related questions become relevant (Crowley et al. 2018; 
National Academies of Sciences, Engineering, and Medicine, 
2016). For example, understanding the costs of serving an 
additional participant once staff are trained and infrastructure 
is in place can be informative, particularly because such costs 
are often substantially lower than average costs. When costs 
have been estimated using the Ingredients Method (Levin 
et al. 2012), these marginal or incremental costs may be read- 
ily estimable by summing the cost of ingredients that vary 
with, or are needed to serve, the extra participant, and ignoring 
costs that are fixed. 

Cost estimates have also been called for to support 
broader dissemination of efficacious interventions 
(Gottfredson et al. 2015), yet many scale implementa- 
tion studies do not address costs (see, e.g., Kozica et al. 
2016; Moessner et al. 2016; Olweus et al. 2019; 
Velasco et al. 2015). Although estimates of the marginal 
cost per participant can inform decisions to expand pro- 
gram delivery over a small range (Alfonso et al. 2019), 
they are generally insufficient to estimating costs at 
scale as fewer costs are fixed and margins beyond the 
individual participant need to be considered. For exam- 
ple, expanding delivery of FSN to large numbers of 
additional students at new sites is likely to necessitate 
additional capacity building and infrastructure invest- 
ment. In this case, the marginal costs of bringing new 
staff and schools on board need to be included in the 
analysis alongside marginal participant costs. When in- 
terventions are disseminated broadly, differences in 
wages, characteristics of the population served, number 
of implementing sites, and staff turnover further add to 
the complexity of estimating costs at scale (Brabson et 
al. 2019; O’Connor et al. 2018). 


The Present Study 


To our knowledge, cost studies of effective early intervention 
programs have not estimated total and average costs, incre- 
mental costs per participant, and broader dissemination costs, 
including attention to sources of variability in scale estimates, 
within a single study. This study used the Ingredients Method 
(Levin and McEwan 2001) to estimate each of these costs in 
relation to FSN, thereby increasing the utility of intervention 
cost information for stakeholders. First, we estimated the total 
and per student costs of implementing FSN for 2 years, in 
preschool and kindergarten, using data from our current, on- 
going efficacy trial of FSN. Next, we estimated the incremen- 
tal (or marginal) cost of FSN, or the cost of adding one student 
within a school or preschool center after staff are trained and 
the program has been implemented. We then used this infor- 
mation to estimate the cost to disseminate FSN to 40 students 
within one district. Finally, we show how dissemination cost 
estimates are sensitive to several real-world cost drivers. 


Method 


The study in which this cost analysis took place examined the 
efficacy of the FSN intervention for improving behavioral and 
academic outcomes to support learning in preschool and kin- 
dergarten settings. We conducted the research in Head Start, 
pre-k, and kindergarten programs in Kentucky, Oregon, and 
Illinois. After recruiting participants through procedures de- 
scribed below, school sites were randomly assigned to FSN or 
a business as usual comparison group. The cost analysis used 
data from the 8 Kentucky intervention sites, which offered 
preschool and kindergarten follow-up intervention to two co- 
horts of students who were one year apart in school during the 
2015-2016 through the 2017-2018 school years. 


Participants 


The eight Kentucky sites (two early childhood centers, six 
elementary schools) had two to six preschool teachers each 
(M = 3.6, SD= 1.6). Across the sites, 29 teacher-child-parent 
triads participated in FSN in two cohorts. Children had a mean 
age of 4.1 years (SD=0.3) and were predominately African 
American (69%) and male (70%). Mean aggressive and mal- 
adaptive behavior screening scores denoted moderate or se- 
vere disruptive behavior. 


FSN Implementation 
Pre-intervention Recruitment and Screening 


The principal investigator and research managers (referred to 
as “FSN managers” from here forward) worked with Jefferson 
County Public School administrators to identify participating 
early childhood centers. School administrators from those 
sites were contacted and, if they agreed, a meeting with po- 
tential teachers was held to describe the study. Teachers who 
consented to participate in the study subsequently distributed 
waiver of consent letters to parents. The letters described the 
study’s purpose and the class-wide screening procedure. The 
FSN managers trained teachers to complete the Early 
Screening Procedure, which involved completing surveys to 
identify children at elevated risk for school failure based on 
teacher-observed externalizing problem behaviors. Study staff 
scored teacher screenings (5 students per classroom) and 
ranked students in order of risk. They then contacted parents 
of the highest ranked child in each classroom and invited them 
to participate. If the parents agreed, staff set up a meeting to 
obtain informed consent. If they declined, parents of the next 
highest ranked child were invited to participate. We repeated 
this process until we obtained parent consent for one eligible 
child in each classroom or all families of eligible children 
declined. Eighty-five percent of parents contacted agreed to 
participate. 
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Coach and Teacher Training 


Coaches and teachers were trained separately. Coaches were 
trained at a 2-day workshop (5.5 h on day 1, 4 h on day 2) 
offered in a district resource center. Each coach was then 
assigned to a non-participating early childhood classroom, 
where they spent 3 to 4 weeks practicing the intervention. 
They were also monitored closely by the FSN managers, 
who were present daily at the training site to debrief before 
and after coaches went into classrooms. The FSN managers 
also observed coaches regularly and offered feedback. 
Teachers who were randomized to the intervention group par- 
ticipated in a 2-h training session prior to implementation. 
Additional details can be found in Walker et al. (2018). 


Intervention 


After parental consent was obtained, the intervention was 1m- 
plemented in the preschool year as described in the introduc- 
tion to this paper. A kindergarten booster phase developed for 
the efficacy trial occurred during the initial 2 or 3 months of 
the following year to assist the child in maintaining his or her 
behavioral gains. Kindergarten teachers were invited and 
consented to participate. Activities were like those offered in 
the preschool year but abbreviated in that the green card game 
was played for a shorter duration and the number of days in 
each phase was reduced. Twenty-seven of 29 students (93%) 
received booster intervention services as two kindergarten 
teachers agreed to data collection but not to implementing 
FSN. 


Ongoing Support 


Coaches attended an hour per week group supervision meet- 
ing with the FSN managers, throughout the intervention. The 
purpose of support is primarily to trouble shoot challenging 
cases and share successes. 


Data Sources 


The Ingredients Method requires data about the quantity and 
unit prices of all key ingredients or resources used to carry out 
the intervention, which are multiplied together to determine 
cost. Resources financed directly (e.g., coach labor, supplies) 
and indirectly (e.g., teacher time, overhead) need to be includ- 
ed for cost estimates to be comprehensive and reflect the op- 
portunity cost of delivering FSN. Data sources for each FSN 
key ingredient are described below. 


Personnel Wage and Fringe Benefit Rates 


For the FSN managers, annual salaries were divided by 2080, 
the number of hours in a work year, to determine an hourly 
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wage; fringe benefit rates were from project budget records. 
Hourly wage and fringe benefit rates for coaches, who were 
masters level professionals (e.g., school social workers, for- 
mer educators) and part-time itinerant employees of the 
University of Louisville, came from payroll records. Teacher 
wages were estimated from the Bureau of Labor Statistics 
(BLS) Occupational Employment Statistics for the state of 
Kentucky, Occupation code 25-2012, Kindergarten 
Teachers, Except Special Education (United States 
Department of Labor 2018a). To teach preschool in 
Kentucky, teachers must be certified in interdisciplinary early 
education, birth to primary, through the Kentucky Department 
of Education. Thus, kindergarten and preschool teachers are 
on the same pay scale. Zeacher fringe benefit rates were from 
BLS Employer Costs for Employee Compensation (United 
States Department of Labor 2018b). Sensitivity analyses de- 
scribed below used the same fringe benefits, but Kentucky 
wage rates varied with staff credentials: (a) coaching by para- 
professionals (occupational code 25-9041) or a school psy- 
chologist (occupational code 19-3031) and (b) intervention 
oversight by a special education teacher (occupational code 
25-2052) or school psychologist. 


Personnel Hours 


Coaching was the most resource-intensive personnel position. 
Hours spent by coaches came directly from hourly time sheets 
submitted prospectively on a bi-monthly basis throughout 
their involvement in the study. We used notes from supervi- 
sion meetings, where interventionists were assigned to stu- 
dents, to identify intervention start and stop dates for each 
coach and then allocated hours across major activities (pre- 
intervention, intervention, and support). Any hours incurred 
prior to the intervention start date were allocated to pre-inter- 
vention. Hours from the coach’s first to last intervention dates 
were assigned to intervention and support activities. One hour 
per week was allocated to support, the rest to intervention. 

Hours spent by the FSN managers on recruitment, screen- 
ing, and training were estimated retrospectively from project 
records and outlook calendars showing these activities. We 
allocated 1 h per week for the duration of the intervention 
period to ongoing support activities. 

Teachers were involved in recruitment, screening, training, 
intervention, and support activities. Although we attempted to 
have all teachers complete logs showing time spent on FSN 
activities, most were resistant to this activity, and the data were 
too incomplete to be usable. Thus, average pre-intervention 
time estimates were based on FSN manager knowledge of 
FSN (preschool recruitment | h; screening 45 min; training 
2 h). Conversations with coaches indicated teacher support 
took approximately 3 h per teacher. Discussions with pre- 
school teachers indicated they spent approximately 30 min 
each day when the intervention was implemented, or 10 
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intervention hours total per teacher. Kindergarten teachers re- 
ported less time on intervention-related activities: 1 h for re- 
cruitment and 2.5 h on the intervention. These estimates are in 
line with the few logs we received; limitations in teacher time 
estimates are discussed under study limitations. 


Supplies 


This category included supplies needed for pre-intervention 
(teacher recruitment packets, student screening kits, student 
screening surveys) and intervention (FSN preschool kits and 
refills, FSN kindergarten materials, intervention timers, sup- 
plemental books) activities, as well as food and snacks pro- 
vided for teacher training sessions. The amounts and unit costs 
came from project expense records and university procard/ 
credit card receipts. Though new to FSN, kindergarten booster 
materials are available from the publisher upon request. 


Overhead 


Because recommendations for including overhead in cost es- 
timates differ (National Academies of Sciences, Engineering, 
and Medicine 2016; Education Endowment Fondation 2018), 
we estimated costs with and without overhead at 20% of per- 
sonnel costs. 


Inflation 


We used the Implicit Price Deflator for Personal Consumption 
Expenditures (Economic & Analysis) to adjust all nominal 
unit prices to constant 2015 dollars, the year in which the 
intervention started. This involved multiplying 2016 and 
2017 nominal prices (i.e., prices observed in years 2 and 3 
of the intervention) by the 2015 quarter three index value 
(coinciding with the start of the school year) divided by the 
nominal quarter three index value. Adjusting to constant 2015 
dollars allowed us to combine data from the two cohorts so 
that we had a larger sample on which to base cost estimates 
(N=29; n= 14 in cohort 1, n=15 in cohort 2) and to add 
values across two intervention years. 


Analysis Plan 


Our objectives were to estimate the total and average cost per 
student of FSN as implemented in the efficacy study, the in- 
cremental cost of delivering FSN to one more student, and the 
cost of disseminating FSN to 40 students within one school 
district. 


Total Costs and Average Costs per Student 


We calculated the total cost of implementing the FSN inter- 
vention with 29 triads by summing the costs of key ingredients 


for all major activities: pre-intervention (recruitment, screen- 
ing, and training), intervention, and support. Because person- 
nel time and associated costs typically drive the overall costs 
of social programs, we also reported the hours devoted by staff 
to each major activity. We calculated the average cost per 
student by dividing the total FSN cost by the 29 students 
served in the two cohorts. 


Incremental Cost of Serving One More Student 


We first identified which resources would vary if one addi- 
tional student received FSN, versus which would remain in- 
variant or fixed. We assumed the additional student would be 
at a school already participating in the intervention and that a 
trained coach would be assigned; thus, no additional school 
recruitment or coach training costs would be incurred. 
Because coach support occurred in a group, no additional 
support resources would be required, and so coach support 
costs would also be fixed. Adding one more student, however, 
would necessitate additional classroom screening, teacher 
training in preschool, intervention delivery by coach and 
teacher during the preschool and kindergarten years, and 
teacher support. Incremental personnel resources (e.g., hours 
per student) were the averages per student shown in Table 2 
and were multiplied by the relevant wage and fringe rate for 
that personnel category to determine incremental personnel 
costs. Incremental supplies consisted of one teacher recruit- 
ment packet, one set of screening surveys, and a super student 
book; other supplies that were used by coaches for multiple 
students, like intervention timers and the FSN kit, would re- 
main fixed. Supply costs are described above. Incremental 
overhead costs were estimated at 20% of incremental person- 
nel costs. The total incremental cost per student was estimated 
by summing the costs of all incremental (or variable) 
resources. 


Cost of Disseminating FSN to 40 Students in One School 
District 


To estimate the cost of disseminating the intervention in an 
authentic educational setting, we again used cost analysis re- 
sults as a foundation. Our dissemination scenario involved 
offering 2 years of FSN to a single cohort of 40 students, 
whom we assumed attended 10 sites, each with four partici- 
pating classrooms. A special education resource teacher 
would serve as the FSN manager, and school social workers 
would implement the intervention to students. We assumed a 
university faculty member or similarly qualified professional 
would conduct initial training and provide some consultation 
support. We also assumed students would stay in the same 
school for preschool and kindergarten, and there would be 
no coach turnover during the two intervention years. 
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We developed a dissemination cost model that accounted for 
the cost of key ingredients used in each of the major activities 
reported previously. For each ingredient, we first determined the 
unit over which the cost varied. We then multiplied the unit cost 
estimated in the efficacy trial by the number of units needed in 
the dissemination scenario. For example, we multiplied the train- 
ing cost per coach in the efficacy trial by 10 coaches to be trained 
in the dissemination effort. As a second example, coach interven- 
tion costs varied with the number of students served. We multi- 
plied the coach intervention cost per student in the efficacy trial 
by 40 students served in dissemination. We worked through each 
key ingredient similarly and calculated costs with and without 
overhead. We summed the elements to calculate a total cost to 
disseminate FSN to 40 students and divided by 40 to estimate the 
average dissemination cost per student. 


Sensitivity Analysis 


Because many different factors can affect real-world implemen- 
tation costs (Brabson et al. 2019; O’Connor et al. 2018), we 
examined the effect of four cost drivers on the dissemination 
costs calculated above. Sensitivity scenario | varied the number 
of schools and coaches needed to serve 40 students from a low of 
8 schools/coaches to a high of 12 schools/coaches; fewer schools 
and coaches would be more cost efficient. Scenario 2 varied 
coach turnover, which had been nil in the base case to a lower 
rate of 30% and a higher rate of 50% between the preschool and 
kindergarten years. Scenario 3 varied the credentials of the 
coaches and managers, with the lower cost option utilizing para- 
professionals as coaches and school social workers as managers 
and the higher cost option utilizing school psychologists in both 
roles. Scenario 4 varied student risk and mobility. Higher risk 
necessitated more intervention time per student (+ 15% com- 
pared to the base case) while lower risk necessitated less (— 
15% compared to the base case). Low student mobility was 
defined as no loss to kindergarten follow-up (as in the base case) 
whereas higher mobility was defined by a 15% loss of students 
served in kindergarten. Because mobility and risk tend to be 
positively correlated, the two options in scenario 4 were low 
need/low mobility and high need/high mobility. To conduct the 
sensitivity analyses, we adjusted relevant inputs to the dissemi- 
nation cost model (e.g., personnel wages under scenario 3), re- 
estimated costs, and compared them to the base case. Unlike the 
other analyses, all sensitivity analyses modeled costs with over- 
head to keep comparisons more manageable. 


Results 
Total and Average Costs per Student 


The total and average costs per student (in constant 2015 
dollars) to implement the FSN intervention with 29 triads 
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are reported in Table 1. Intervention costs for the preschool 
and kindergarten year totaled $125,556 with overhead 
($105,634 without), or $4330 per student ($3643 without 
overhead). Most of the investment (80%) occurred in the pre- 
school year, with 20% in kindergarten. Fifty-five percent of 
the costs incurred were for intervention activities, while pre- 
intervention activities accounted for 34%. At 11% of total 
costs, ongoing support was a relatively small investment. 
Personnel comprised the largest share of total costs, account- 
ing for 80% of the total, followed by overhead at 16%. 

To better understand investments in personnel and allow 
potential adopters to estimate costs of their own staff, we 
examined the hours spent by different personnel types (1.e., 
managers, coaches, teachers) on pre-intervention, interven- 
tion, and support activities in each of the 2 years of interven- 
tion. Estimates are reported in Table 2. Coach time accounted 
for 72% of all personnel hours, with managers accounting for 
12% and teachers accounting for 17%. Fifty-nine percent of 
all hours were incurred in providing the intervention, with 
31% going to pre-intervention activities and 10% to support. 
Training accounted for 25% of personnel costs. Managers 
spent the greatest share of their time on pre-intervention ac- 
tivities, while the greatest share of coach and teacher hours 
were spent delivering the intervention. 


Incremental Costs 


Table 3, available online, summarizes the incremental cost of 
serving an additional student once an intervention infrastruc- 
ture (e.g., trained coaches, fixed supplies, ongoing support 
system) has been established. At $2287 ($1911 without over- 
head) for preschool and $683 ($569 without overhead) for 
kindergarten, the total cost per additional student is $2970 
($2480 without overhead). This is 31% lower than the average 
per student cost and highlights the significant efficiencies to 
be gained once FSN infrastructure is in place. Importantly, and 
understandably since existing infrastructure reduces the need 
for additional training and ongoing support on the margin, an 
even larger share of incremental costs is in support of direct 
intervention services to students, 77% compared to 55% of 
average costs. 


Dissemination Costs 


The total cost (2015 dollars) of serving 40 students in 10 early 
childhood sites was estimated to be $123,900 with overhead 
($104,499 without) during preschool and $46,205 with over- 
head ($38,664 without) during kindergarten, for a total of 
$170,106 with overhead ($143,163 without). The per student 
cost for 2 years of intervention was $4253 when overhead was 
included, and $3579 when it was not. Incremental costs were 
estimated at $2970 with overhead ($2480 without). Like the 
actual cost analysis, most of the investment, 73%, occurred in 
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Table 1 Total and average costs per student: First Step Next (2015 dollars) 
Total FSN costs Average FSN cost per student 
Pre- Intervention Support Total® Pre- Intervention Support Total Share of cost by 
intervention intervention resource? 
Preschool 
Personnel $30,866 $39,700 $8190 $78,756 $1064 $1369 $282 $2716 78.6% 
Equipment $0 $0 $0 $0 $0 $0 $0 $0 0.0% 
Supplies $1377 $3902 $0 $5279 $47 $135 $0 $182 5.3% 
Other $394 $0 $0 $394 $14 $0 $0 $14 0.4% 
Overhead $6173 $7940 $1638 $15,751 $213 $274 $56 $543 15.7% 
Total $38,810 $51,543 $9828 $100,180 $1338 $1777 $339 $3454 
Share of cost® 38.7% 51.5% 9.8% 
Cost minus overhead $32,637 $43,603 $8190 $84,429 $1125 $1504 $282 $2911 
Kindergarten 
Personnel $2961 $14,251 $3644 $20,856 $102 $491 $126 $719 82.2% 
Equipment $0 $0 $0 $0 $0 $0 $0 $0 0.0% 
Supplies $0 $349 $0 $349 $0 $12 $0 $12 1.4% 
Other $0 $0 $0 $0 $0 $0 $0 $0 0.0% 
Overhead $592 $2850 $729 $4171 $20 $98 $25 $144 16.4% 
Total $3553 $17,451 $4373 $25,376 =$123 $602 $151 $875 
Share of cost 14.0% 68.8% 17.2% 
Cost minus overhead $2961 $14,600 $3644 $21,205 $102 $503 $126 $731 
Total 
Personnel $33,827 $53,952 $11,834 $99,612 $1166 $1860 $408 $3435 79.3% 
Equipment $0 $0 $0 $0 $0 $0 $0 $0 0.0% 
Supplies $1377 $4252 $0 $5629 $47 $147 $0 $194. 45% 
Other $394 $0 $0 $394 $14 $0 $0 $14 0.3% 
Overhead $6765 $10,790 $2367 $19,922 $233 $372 $82 $687 15.9% 
Total $42,363 $68,993 $14,200 $125,556 $1461 $2379 $490 $4330 
Share of cost 33.7% 55.0% 11.3% 
Cost minus overhead $35,597 $58,203 $11,834 $105,634 $1227 $2007 $408 $3643 


“Preschool costs were 80% and kindergarten costs were 20% of the total spent across two intervention years 


> The share of costs by resource is calculated by dividing the resource cost in a given year by the total FSN cost for that year 


“The share of cost by activity (e.g., pre-intervention, intervention) is calculated as the activity cost divided by total FSN cost in a given year 


the preschool year, with 27% in kindergarten. The share of 
costs going to different resources and activities was similar to 
the actual cost analysis, with personnel and intervention costs 
remaining the major cost drivers. These costs are summarized 
in Table 4, available online. 

Sensitivity analyses systematically varied four cost drivers 
and assessed the implications for total, average, and incremen- 
tal dissemination costs: (a) the number of schools and coaches 
needed to serve 40 students (scenario 1); (b) coach turnover 
rates (scenario 2); (c) coach and site management credentials 
(scenario 3); and (d) student characteristics (scenario 4). 
Because we had low- and high-cost options for each scenario, 
sensitivity analyses produced a range of total dissemination 
costs. For simplicity, all estimates included overhead. Results 
displayed in Fig. 1 show that FSN cost estimates were indeed 


sensitive to these cost drivers. Although the average base cost 
per student was $4253, the least and most expensive scenarios 
were $3141 (reflecting low-cost inputs across all scenarios) 
and $7829 (reflecting high-cost inputs across all scenarios). 
Incremental costs varied from $2216 to $4749. Figure 1 also 
illustrates that variation in intervention costs, rather than pre- 
intervention and support costs, accounts for most the variation 
in dissemination cost estimates. 

Figure 2 provides a different picture of the dissemination 
sensitivity analysis. Focusing on average cost per student, it 
shows pre-intervention, intervention, support, and total costs 
per student for the base case and the four scenarios. Both low- 
cost and high-cost options are displayed. Figure 2 shows that 
average costs per student varied from a low of $3634 per 
student, a savings of $600 per student compared to the base 
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Table 2. Average personnel hours per student by personnel type and intervention activity 


Pre-intervention activities 


Recruitment Screening Training 
Preschool 
Program managers 2.2 1.8 5.7 
Coaches 0.0 0.0 22.9 
Teachers 1.0 0.8 1.5 
Total 3.2 2.5 30.1 
Share of hours® 3.4% 2.6% 31.2% 
Kindergarten 
Program managers 0.7 0.0 0.1 
Coaches 0.0 0.0 0.5 
Teachers 0.8 0.0 0.0 
Total 1.5 0.0 0.6 
Share of hours 5.8% 0.0% 24% 
Total 
Program managers 29 1.8 5.8 
Coaches 0.0 0.0 23.5 
Teachers 1.8 0.8 1.5 
Total 4.7 25 30.7 
Share of hours 3.9% 2.1% 25.2% 


Prev Sci 


Total Intervention Support Total" Share of hours by 
personnel type? 

9.7 0.0 1.6 11.3 11.7% 

22.9 42.1 4.0 69.0 71.5% 

3.2 10.0 3.0 16.2 16.8% 

35.9 52.1 8.6 96.5 

37.2% 54.0% 8.9% 79.0% 

0.8 0.0 1.1 19 74% 

0.5 17.6 3.0 21.1 82.5% 

0.8 1.8 0.0 2.6 10.2% 

2.1 19.4 4.1 25.6 

8.2% 75.7% 16.1% 21.0% 

10.5 0.0 2.7 13.2 10.8% 

23.5 59.7 7.0 90.2 73.8% 

4.0 11.8 3.0 18.8 15.4% 

38.0 71.5 12.7 122.2 

31.1% 58.5% 10.4% 


“Preschool personnel hours were 78% and kindergarten personnel hours were 21% of the total hours across two intervention years 


> The share of hours by personnel type is calculated by dividing the personnel hours in a given year by the total FSN hours for that year 


“The share of hours by intervention activity (e.g., pre-intervention, intervention) is calculated by dividing the activity hours in a given year by the total 


FSN hours for that year 


case, to a high of $6817 per student, an increase of $1564 
above the base case. Like Fig. 1, Fig. 2 shows that intervention 
costs drive variability in dissemination costs. Figure 2 also 
shows that dissemination costs were very sensitive to staff 
credentials and associated wage rates (scenario 3), reflecting 
that personnel costs make up the clear majority of FSN costs. 
They were somewhat sensitive to differences in student mo- 
bility and risk (scenario 4), which ultimately affect interven- 
tion costs, but were not very sensitive to variation in the num- 
ber of implementing sites and teachers or to staff turnover, 
which affect pre-intervention and support costs but not inter- 
vention costs. 


Discussion 


The long-term societal costs for youth with disruptive behav- 
ior disorders are well documented (Aos et al. 2004), yet com- 
prehensive cost analyses that inform high-quality implemen- 
tation capable of achieving impact are scarce (National 
Academies of Sciences, Engineering, and Medicine 2016; 
Tran et al. 2018). The current study helps to fill this gap, 
providing a systematic and replicable comprehensive cost 
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analysis of FSN, an empirically based early intervention of- 
fered at the beginning of a child’s school career to improve 
school success. 

The study’s total and average cost estimates are quite compre- 
hensive. They are also rare in that they include training to imple- 
mentation proficiency (33% off the total cost), ongoing support 
(12% of the total cost), teacher salary costs (17% of the personnel 
costs), and overhead (20% of personnel costs). At $4330 (2015 
dollars, $3643 without overhead) in average cost per family for 
2 years of intervention, FSN costs are also well-aligned with two 
other evidence-based interventions for children with disruptive 
behavior disorders, the Incredible Years and Parent-Child 
Interaction Therapy described in this paper’s introduction. 
Similar to other prevention and early intervention programs that 
are labor intensive and require little in terms of supplies and/or 
equipment (e.g., Menting et al., 2013; Zisser and Eyberg 2017), 
our results demonstrated that approximately 80% of FSN costs 
were driven by personnel hours. 

In response to calls for greater utility and informational 
content in economic evaluations of early intervention and pre- 
vention programs, including cost analyses (National 
Academies of Sciences, Engineering, and Medicine 2016), 
this study goes beyond most extant studies in several ways. 
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Table 3 Incremental cost per student (2015 dollars) 


Incremental hours or units* Incremental costs” 
Unit cost Pre- Intervention Support Total Pre- Intervention Support Total® Share of 
intervention intervention cost® 
Preschool 
Personnel 
Manager/supervisor $41.18 65 6.5 $267 $267 
Coach $23.67 42.1 42.1 $997 $997 
Teacher $38.05 3.2 10.0 3.0 16.2 $123 $380 $114 $618 
Sub-total personnel 9.7 52.1 3.0 64.8 $390 $1377 $114 $1881 82.3% 
Supplies 
Screening survey $2.98 1.0 $3 $3 
Teacher recruitment packet $1.49 1.0 $1 $1 
Super student book $25.00 1.0 $25 $25 
Sub-total supplies $3 $26 $0 $29 1.3% 
Overhead $78 $275 $23 $376 = 16.5% 
Total—preschool 9.7 52.1 3.0 64.8 $471 $1679 $137 $2287 
Cost minus overhead $393 $1404 $114 $1911 
Kindergarten 
Personnel 
Manager/supervisor $51.54 0.7 0.7 $36 $36 
Coach $24.70 17.6 17.6 $435 $435 
Teacher $38.06 0.8 1.8 0.0 2.6 $30 $69 $99 
Sub-total personnel 1.5 19.4 0.0 20.9 $66 $503 $0 $569 83.3% 
Supplies $0 0.0% 
Overhead $13 $101 $0 $114 16.7% 
Total—kindergarten 1.5 19.4 0.0 20.9 $79 $604 $0 $683 
Cost minus overhead $66 $503 $0 $569 
Total 
Personnel 12 71.5 3.0 85.7 $456 $1881 $114 $2451 82.5% 
Supplies (preschool only) $3 $26 $0 $29 1.0% 
Overhead $91 $376 $23 $490 =16.5% 
Total $550 $2283 $137 $2,970 
Share of total® 13.1% 83.4% 3.5% 18.5% 76.9% 4.6% 
Cost minus overhead $459 $1907 $114 $2480 


This table shows the cost of the resources that vary when an additional student is served: personnel: in the preschool year, the student is screened and the 
teacher is trained. In kindergarten, the teacher is recruited. The intervention is conducted in both years by a coach who has been previously trained. The 
coach is supported by activities that are already taking place; there is no additional cost. Unit costs reflect weighted average hourly wage and fringe 
benefit rates for a given personnel category. They are derived by dividing the category-specific personnel expenditure per student by the category- 
specific hours per student. Supplies: the only supplies needed are a teacher recruitment packet, student screening surveys, and a super student book. Unit 
costs reflect the cost of 1 survey, 1 teacher recruitment packet, or 1 super student book. Overhead: 20% of personnel costs 


“We assumed hours spent in incremental pre-intervention, intervention, and support activities would equal the average reported in Table 2; for FSN 
managers, hours reported here are less than Table 2 because some hours reported in Table 2 were for fixed activities (e.g., school recruitment, coach 
training). Incremental supply units always equal to 1 


> Incremental costs are the product of the unit cost and incremental hours or units 
“Preschool costs were 80% and kindergarten costs were 20% of the total spent across two intervention years 
“The share of costs by resource is calculated by dividing the resource cost in a given year by the total FSN cost for that year 


° The share of costs by intervention activity (e.g., pre-intervention, intervention) is calculated by dividing the activity cost in a given year by the total FSN 
cost for that year 
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Table 4 _ First Step Next dissemination costs (2015 dollars) 


Total dissemination costs 
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Average dissemination cost per student 


Pre- Intervention Support Total" Pre- Intervention Support Total Share of cost by 
intervention intervention resource” 
Preschool 
Personnel $32,945 $54,759 $9300 $97,004 $824 $1369 $232 $2425 78.3% 
Supplies $756 $6275 $0 $7030 $19 $157 $0 $176 5.7% 
Other $465 $0 $0 $465 $12 $0 $0 $12 0.4% 
Overhead $6589 $10,952 $1860 $19,401 $165 $274 $46 $485 15.7% 
Total FSN costs $40,754 $71,986 $11,160 $123,900 $1019 $1800 $279 $3098 
Share of cost by 32.9% 58.1% 9.0% 72.8% 
intervention activity® 
Cost minus overhead $34,165 $61,1034 $9300 $104,499 $854 $1526 $232 $2612 
Kindergarten 
Personnel $1207 $34,059 $2443 $37,709 $30 $851 $61 $943 81.6% 
Supplies $0 $955 $0 $955, $0 $24 $0 $24 2.1% 
Other $0 $0 $0 $0 $0 $0 $0 $0 0.0% 
Overhead $241 $6812 $489 $7542 $6 $170 $12 $189 16.3% 
Total FSN costs $1448 $41,825 $2932 $46,205 $36 $1046 $73 $1155 
Share of cost by 3.1% 90.5% 6.3% 27.2% 
intervention activity 
Cost minus overhead $1207 $35,013 $2443 $38,664 $30 $875 $61 $967 
Total 
Personnel $34,152 $88,818 $11,743 $134,713 $854 $2220 $294 $3368 79.2% 
Supplies $756 $7229 $0 $7985 $19 $181 $0 $200 4.7% 
Other $465 $0 $0 $465 $12 $0 $0 $12 0.3% 
Overhead $6830 $17,764 $2349 $26,943 $171 $444 $59 $674 15.8% 
Total FSN costs $42,203 $113,811 $14,092 $170,106 $1055 $2845 $352 $4253 
Share of cost by 24.8% 66.9% 8.3% 
intervention activity 
Cost minus overhead $35,373 $96,047 $11,743 $143,163 $884 $2401 $294 $3579 


Major assumptions: FSN is disseminated to 40 students at 10 schools in 1 school district. Students stay in the same school in both intervention years. 
They are served by 10 coaches in the 2 years. The intervention is managed by a school social worker, who provides ongoing support. The school social 


worker is trained with intervention coaches 


“Preschool costs were estimated to account for 82% and kindergarten costs 18% of total dissemination costs across two intervention years 


> The share of costs by resource is calculated by dividing the resource cost in a given year by the total FSN cost for that year 


“The share of costs by intervention activity (e.g., pre-intervention, intervention) is calculated by dividing the activity cost in a given year by the total FSN 


cost for that year 


First, the study’s incremental cost estimate of $2970 per addi- 
tional student served ($2480 without overhead) demonstrates 
efficiencies obtained once an infrastructure is in place. 
Specifically, the per student cost once FSN staff are trained 
and ongoing support has been established was roughly a third 
less than the average cost, with most of the incremental re- 
sources going directly towards intervention activities, impor- 
tant information for consumers. 

Second, with greater need for translation and dissemination 
of effective interventions (Gottfredson et al. 2015; Proctor 
et al. 2011; Spoth et al. 2013), this study used total and aver- 
age cost estimates as the foundation for estimating the costs of 
disseminating FSN to 40 students in one district. The analysis 
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clearly showed that the most important drivers of dissemina- 
tion costs were wage and fringe benefit rates for personnel, 
particularly personnel costs associated with direct interven- 
tion. Student risk and mobility were a distant second in terms 
of impact on dissemination costs, and the impact was due to 
their implications for intervention time by coaches. Other 
sources of variability, such as staff turnover and economies 
of scale with respect to the number of schools and coaches 
required to serve 40 students, had far less impact on dissem- 
ination costs. These findings suggest that optimizing 
personnel-heavy interventions like FSN may be possible 
through high-quality training of lower wage personnel. 
Although not examined in this study, FSN’s structured 
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Base Case Total Cost pee ‘ase Average Cost repens Incremental Cost 
7 $4, 253per —~_ 52.970 per i 
$170,106 o Student ~~. Student — 


$2,216 


Total 


$125,658 a $313,159 $3,141 
$32,383 | $85,592 
$82,725 pane) $206,577 $2,068 = $5,164 
$10,549 ; $20,990 sx $525 


*Key assumptions: Dissemination Base Case. 40 students served by 10 coaches in 10 schools; special education teacher provides site-based management. 
Other inputs (e.g., coach credentials and wages, intervention time, supplies) are as in efficacy trial. Scenario 1. Lower cost: 8 schools/coaches (“economies of 
scale”). Higher cost: 12 schools/coaches. Scenario 2. Lower cost: 30% coach turnover. Higher cost: 50% coach tumnover. Scenario 3: Lower cost: Coaches are 
paraprofessionals, site-based management by school social worker. Higher cost: School psychologists serve as coaches and site-based managers. Scenario 4: 
Lower cost: -15% intervention time due to lower risk; also low mobility, meaning no loss to kindergarten follow-up. Higher cost: +15% intervention time due 
to higher risk; 15% loss of students served to kindergarten follow-up. 
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$433 | $575 
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Fig. 1 Dissemination cost sensitivity analysis—variation by major activity (2015 dollars) 


training and manualized intervention approach, coupled with 
ongoing support, may indeed allow lower-cost personnel to 


deliver the intervention with fidelity and achieve improved 
outcomes for youth with disruptive behavior disorders. 
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High Cost Options 
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“Key assumptions: Dissemination Base Case. 40 students served by 10 coaches in 10 schools; special education teacher provides site-based 
management. Other inputs (e.g., coach credentials and wages, intervention time, supplies) are as in efficacy trial. Scenario 1. Lower cost: 8 
schools/coaches (“economies of scale’). Higher cost: 12 schools/coaches. Scenario 2. Lower cost: 30% coach turnover. Higher cost: 50% 
coach turnover. Scenario 3: Lower cost: Coaches are paraprofessionals, site-based management by school social worker. Higher cost: School 
psychologists serve as coaches and site-based managers. Scenario 4: Lower cost: -15% intervention time due to lower risk; also low mobility, 
meaning no loss to kindergarten follow-up. Higher cost: +15% intervention time due to higher risk; 15% loss of students served to kindergarten 


follow-up. 


Fig. 2. Dissemination cost sensitivity analysis: average cost per student in 
relation to key cost drivers (constant 2015 dollars). Key assumptions: 
dissemination base case. Forty students served by 10 coaches in 10 
schools; special education teacher provides site-based management. 
Other inputs (e.g., coach credentials and wages, intervention time, sup- 
plies) are as in efficacy trial. Scenario 1: lower cost: 8 schools/coaches 
“economies of scale”). Higher cost: 12 schools/coaches. Scenario 2: 


lower cost: 30% coach turnover. Higher cost: 50% coach turnover. 
Scenario 3: lower cost: coaches are paraprofessionals, site-based manage- 
ment by school social worker. Higher cost: school psychologists serve as 
coaches and site-based managers. Scenario 4: lower cost: — 15% inter- 
vention time due to lower risk; also low mobility, meaning no loss to 
kindergarten follow-up. Higher cost: + 15% intervention time due to 
higher risk; 15% loss of students served to kindergarten follow-up 
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Examining whether cost efficiencies can be gained without 
sacrificing intervention impact would provide valuable infor- 
mation for FSN dissemination. 

Finally, this study’s emphasis on providing resource use 
alongside intervention cost estimates enhances their utility 
for stakeholders operating in diverse locales with differing 
wage rates and supplies costs (Levin and McEwan 2001; 
National Academies of Sciences, Engineering, and Medicine 
2016). They can turn to resource information provided in 
Tables 2 and 3, substitute local unit costs, and estimate a more 
geographically or locally relevant FSN cost estimate. 


Limitations 


Though there is much value in the FSN cost estimates provid- 
ed in this study, there are also some limitations. The analysis 
was limited to one site in a multi-site study and the number of 
students (NV = 29) was relatively small with a limited range of 
demographic characteristics. Replicating costs in subsequent 
FSN implementation studies could enhance the generalizabil- 
ity of all of our cost estimates, ideally allowing for point and 
interval estimation. 

We were not able to estimate teacher and FSN manager 
time prospectively. Although we tried to support teachers 
in completing prospective time logs, their resistance 
amidst the heavy demands of teaching resulted in retro- 
spective estimates. Fortunately, teacher time and FSN 
manager time, which was also estimated retrospectively, 
appear to be relatively small shares of the total personnel 
time, limiting the effect of imprecise measurement on cost 
estimates. Coach time was measured prospectively 
through time logs completed throughout the intervention, 
an important aspect of our method as coaches represented 
70% of all personnel hours. 

Though parents did not incur direct costs in carrying out 
the home component of FSN, they did give their time to the 
intervention and it is possible that some took time off from 
work to attend meetings with teachers and coaches. 
Although the parent role is not meant to be labor intensive, 
with developers estimating 10-15 min per day, we did not 
collect time, cost, or lost wage data from parents involved 
in this study, an area for future study that would provide an 
even more complete picture of resources used in FSN 
implementation. 

Additionally, sensitivity analyses did not take into consid- 
eration all possible variables and did not estimate costs with- 
out overhead. Intended to be illustrative rather than exhaus- 
tive, their focus on common issues or school district features 
likely to be faced in scale implementation (e.g., wages, turn- 
over, student risk) gives them relevance. 
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Implications and Conclusions 


This type of research has important implications for both pol- 
icy makers and educational leaders. Indeed, cost is a key im- 
plementation outcome and one of the standards for evidence 
related to research on prevention interventions (Flay et al. 
2005; Gottfredson et al. 2015). One possible explanation for 
poor implementation efforts in school settings is that admin- 
istrators adopt programs without providing adequate resources 
to implement them, in part because of a lack of comprehensive 
cost information. Our sensitivity analysis provides valuable 
information to administrators and policy makers regarding 
program adoption decisions. Specifically, it enables detailed 
program planning about job descriptions, qualifications, and 
needed supports for high-quality implementation in authentic 
school settings. It also shows that investments in training and 
support infrastructure, activities that support high-quality im- 
plementation and can enable robust intervention impact, are a 
relatively small portion of the total cost. We would argue they 
should not be omitted. Further, it allows decision-makers to 
weigh the advantages and disadvantages of implementing the 
FSN intervention in preschool only—estimated at $3098 per 
child—or preschool plus the kindergarten follow-up year 
which added roughly $1155 to the per student total cost in 
our dissemination model. 

Without the needed investment, intervention outcomes 
may not always be as robust as in controlled research trials. 
For behaviorally at-risk youth and their families, research 
demonstrating that reductions in disruptive behavior disorders 
improve long-term outcomes and are cost-effective is needed 
to help stimulate investment at scale for effective interven- 
tions. Further research on FSN could add to this by examining 
the relationship between intervention costs and improvement 
in children’s behavior. 
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